Identifying candidate cells using image analysis

ABSTRACT

A method for identifying and enumerating candidate target cells within a biological fluid specimen is described. The method includes obtaining a biological fluid specimen, preparing the biological fluid specimen by staining cell features in the biological fluid specimen, capturing a digital image having a plurality of color channels of the biological fluid specimen, and applying image analysis to the digital image. A computer program product for identifying candidate target cells within a biological fluid specimen is also described. The computer program comprises instructions to cause a processor to carry out the image analysis.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.15/998,990, filed Aug. 20, 2018, which is a continuation of U.S. patentapplication Ser. No. 15/632,707, filed Jun. 26, 2017, which is acontinuation of U.S. patent application Ser. No. 15/476,848, filed onMar. 31, 2017, the entire contents of which are hereby incorporated byreference.

TECHNICAL FIELD

The present disclosure relates to identifying candidate cells, e.g.,circulating tumor cells, in an image of a sample.

BACKGROUND

Circulating tumor cells (CTCs) are cancerous cells, often of epithelialorigin, that have detached from a primary tumor and entered thevasculature or lymphatic system. When CTCs invade the circulation, thesemalignant cells gain access to other organs. After shedding from a solidmass, CTCs may come to rest against a vessel wall and extravasate intosurrounding tissue. Angiogenesis helps establish a new tumor at a sitedistant from the original mass. CTCs thus represent seeds for the growthof additional tumors (metastases).

It is understood that the number of CTCs in peripheral blood isassociated with decreased progression-free survival and decreasedoverall survival in patients with metastatic disease, including breast,colorectal and prostate cancers. CTC detection and enumeration fromblood or other bodily fluid samples can be used to evaluate tumorprognosis and assist in the management of cancer patients.

Tumors shed many cells. It is estimated that 1 million CTCs enterperipheral blood per gram of tumor tissue. Within 24 hours, however,only 0.1% remain viable. Viable CTCs are considered “rare” cells becausethey have been observed in the peripheral blood of cancer patients atvery low concentrations, such as one CTC among 10⁶-10⁷ leukocytes(Sakurai et al., 2016). They are also present against a high backgroundof hematopoietic cells and thus their frequency is on the order of 1-10CTCs per 1 mL of whole blood in patients with metastatic disease (Milleret al., 2010). CTCs are therefore difficult to detect and enumerateaccurately.

Biological staining enhances microscopic image analysis. Certain dyesare used to highlight biological cell features and structures. CTCs havedistinguishing histological features visible under a microscope whenparticular stains are applied.

SUMMARY

In one aspect, a method for identifying candidate target cells within abiological fluid specimen includes obtaining a biological fluidspecimen, preparing the biological fluid specimen by staining cellnuclei in the biological fluid specimen, capturing a digital imagehaving a plurality of color channels of the biological fluid specimen,and applying image analysis to the digital image.

In another aspect, a computer program product for identifying candidatetarget cells within a biological fluid specimen is tangibly embodied ina computer readable medium. The computer program comprises instructionsto cause a processor to carry out the image analysis.

In another aspect, a method for enumerating a target cell populationwithin a biological fluid specimen includes the method of identifyingcandidate target cells, followed by classifying a candidate as a targetcell or a non-target element based on a portion of the imagecorresponding to a remaining identified spatially overlapping firstconnected region and second connected region, and counting any candidateclassified as a target cell, to generate a count value.

In another aspect, a method for determining likelihood of cancer in ahuman subject includes comparing the count value with a statisticallydetermined count of circulating epithelial cells from a group oftumor-free patient controls, and assigning a likelihood of canceroccurrence when the total count exceeds a pre-determined value based onstatistical averages of circulating epithelial cell counts from healthysubjects compared with statistical averages of circulating epithelialcell counts from cancer patients.

Preparing the biological fluid specimen includes staining cell nuclei inthe biological fluid specimen with a first bio-conjugated dye having afirst color and configured to bind nucleic acids in the cell nuclei ofthe target cells and staining cytoskeletal cell features in thebiological fluid specimen with a second bio-conjugated dye having asecond color and configured to bind to cytoskeletal cell features of thetarget cells and staining white blood cells in the biological fluidspecimen with a third bio-conjugated dye having a third color andconfigured to bind to human leukocyte antigens in the biological fluidspecimen.

Applying image analysis includes receiving the digital image,identifying first connected regions of pixels of a minimum firstintensity in a first channel of the plurality of color channels,identifying second connected regions of pixels of a minimum secondintensity in a second channel of the plurality of color channels,determining first connected regions and second connected regions thatspatially overlap, determining an aspect ratio of the spatiallyoverlapping first connected regions and second connected regions basedon a color channel of the plurality of color channels, identifying firstconnected regions and second connected regions that spatially overlapand for which the aspect ratio meets an aspect ratio threshold,determining a second connected region and a third connected region thatspatially overlap, determining an intensity ratio of the spatiallyoverlapping second connected region and third connected region based ontwo color channels of the plurality of color channels, eliminating as acandidate a spatially overlapping first connected region and secondconnected region corresponding to a spatially overlapping secondconnected region and third connected region for which the intensityratio does not meet an intensity ratio threshold, and providing aportion of the image corresponding to a remaining identified spatiallyoverlapping first connected region and second connected region to aclassifier as candidates for classification.

Implementations may include one or more of the following features.

The first color may be blue, the second color may be red or orange, andthe third color may be green.

The first stain or bio-conjugated dye may include DAPI(4′,6-diamidino-2-phenylindol). The second stain or bio-conjugated dyemay include a red or orange fluorescent dye conjugated to ananti-cytokeratin (CK) antibody. The third stain or bio-conjugated dyemay include a green fluorescent dye conjugated to an anti-CD45 antibody,or a combination of a first antibody, anti-CD45, and a second antibodypre-conjugated to a green fluorescent dye and targeting CD45.

The second stain or bio-conjugated dye may include a red or orangefluorescent dye conjugated to an anti-cytokeratin (CK) antibody, or acombination of a first antibody, anti-CK, and a second antibodypre-conjugated to a red or orange fluorescent dye and targeting CK.

In a particular implementation, by way of example, the second stain orbio-conjugated dye may include an ALEXA568®-conjugated anti-cytokeratinantibody, and the third stain or bio-conjugated dye may include ananti-CD45-ALEXA488® antibody or a combination of a first antibody,anti-CD45, and a second antibody pre-conjugated to ALEXA488® andtargeting CD45.

The second stain or bio-conjugated may include an ALEXA568®-conjugatedanti-cytokeratin (CK) that includes monoclonal antibodies that areconjugated to ALEXA568® or a combination of a first antibody, anti-CK,and a second antibody pre-conjugated to ALEXA568® and targeting CK.

Identifying the first connected regions may include identifying firstconnected regions that have a minimum first size, and identifying thesecond connected regions may include comprise identifying secondconnected regions that have a minimum second size. Identifying the firstconnected regions may include identifying first connected regions thathave a maximum first size, and identifying the second connected regionsmay include identifying second connected regions that have a maximumsecond size. Identifying the first connected regions and identifying thesecond connected regions comprises a maximally stable extremal regions(MSER) algorithm.

Identifying the first connected regions may include dividing the digitalimage into a plurality of portions, searching each portion for apotential first connected region, and identifying a new portion of thedigital image centered on a potential first connected region found fromthe search. Determining first connected regions and second connectedregions that spatially overlap may include determining whether aboundary of a second connected region fits inside or overlies a boundaryof the first connected region.

A combination of the first connected region and the second connectedregion may be determined. Determining the aspect ratio may includefinding a major axis that extends between two points that are farthestapart on a boundary of the combination, finding a minor axis thatextends perpendicular to the major axis and between two points that arefarthest apart on the boundary on opposites sides of the major axis, andcalculating a ratio of the minor axis to the major axis. The aspectratio threshold may be 0.4 or less.

A boundary box around the combination may be determined, a first numberof pixels within a boundary of the combination and a second number ofpixels in an extent may be determined, a ratio of the first number ofpixels to the second number of pixels may be determined, and the ratiomay be compared to an extent threshold. The extent threshold may bebetween 0.4 and 0.85. The combination may be a union of the firstconnected region and the second connected region.

Determining the intensity ratio may include determining a first averageintensity of the second connected region, determining a second averageintensity of the third connected region, and determining a ratio of thefirst average intensity to the second average intensity. The spatiallyoverlapping second connected region and third connected region may beeliminated if the ratio is below the threshold. The spatiallyoverlapping second connected region and third connected region may beeliminated if (I2/I3)<1, where I2 is the first average intensity and I3is the second average intensity.

Advantages may include one or more of the following.

Areas within a sample region that are likely to contain candidate cellsof interest can be located automatically. These areas can be flagged forfurther evaluation. This can significantly reduce the number of sampleareas that would otherwise need to be reviewed by a human operator. Suchan automated imaging process for CTC detection and enumeration can aidin predicting disease progression and overall survival during therapy,and could allow for serial monitoring of patient prognosis, leading tomore informed patient care choices.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a schematic illustration of a process of identifying CTCs.

FIG. 2 is a schematic diagram illustrating a staining process.

FIG. 3 is a schematic diagram of a system for identifying CTCs.

FIG. 4 is a flow chart of a computer-implemented process of identifyingcandidate cells in a digital image.

FIG. 5 is a flow chart of a computer-implemented process of identifyingoverlapping “blobs” in the digital image.

FIGS. 6A-6C illustrate a method of determining an aspect ratio.

FIG. 7 illustrates a bounding box surrounding a “blob” in a digitalimage.

FIG. 8 is a flow chart illustrating considerations in determiningwhether an object in an image should be classified as a candidate.

Like reference symbols in the various drawings indicate like elements.

DETAILED DESCRIPTION

Sampling of solid tumors is a routine procedure in cancer diagnostics.Next-generation sequencing now enables sensitive, rapid and low-costdetection and analysis of tumor DNA from cancer cells or its constituentDNA that have strayed beyond their original tissues into fluidcomponents between cells, such as, for example, interstitial fluid,lymph, blood, saliva, cerebral spinal fluid, synovia, urine, feces andother secretions. Cancer cell debris sampled away from a primary tumorcan serve as a marker to monitor disease progression and potentiallyassist in cancer diagnosis before symptoms appear.

The process of identifying CTCs on a sample slide begins with narrowingthe regions of the sample slide where candidate CTCs exist for furthermanual review. CTCs typically measure between 8 μm to 25 μm and atypical sample area is between roughly 50 mm² to 1200 mm². Confirmationof “positive events” by visual inspection of tumor cell morphology orother cell characteristics is necessary. Given the large sample areafrom which CTCs must be located, it is difficult and laborious tomanually identify candidates over an entire slide region, andinefficient for a human operator to evaluate imaged slide samples. Somepositive events can also be missed, especially when candidates in theimages are present in low frequencies. Thus, detection andquantification of CTCs is very challenging. The digital image analysisdescribed herein in conjunction with a high resolution microscope can beused to efficiently identify candidate CTCs of interest.

FIG. 1 is a schematic illustration of an example of a process 100 ofidentifying CTCs. Referring to FIG. 1 , a sample 10 of a biologicalfluid, e.g., a bodily fluid such as blood, lymph, cerebral spinal fluid,saliva, synovia, urine, feces or another secretion, is received from aclinical site (step 115). For example, a doctor may wish to have apatient tested, e.g., to detect a cancer before symptoms appear,diagnose a particular cancer, monitor the progression of a cancer orcharacterize the DNA of cancer cells in order to select appropriatetreatment options.

The patient's blood or other bodily fluid sample can be collected at thedoctor's office or at a medical clinic, and the sample sent to theoperator of the system 200. In other implementations, the sample may becollected at the site of the system 200.

The sample can also be subjected to an enrichment process (step 120through step 125). Enrichment of a sample for cancer cells is especiallyuseful when blood samples are being evaluated.

Several CTC enrichment technologies exist to reduce the total number ofcells that must be analyzed. Examples include antibody-functionalizedmicrofluidic devices, cell-size based filtration, passive cell sorting,and immunomagnetic isolation. Other methods, compositions and systemsfor isolating cancer cells of interest include those described inPCT/US2015/023956, which claims the benefit of U.S. ProvisionalApplication No. 61/973,348, filed Apr. 1, 2014 and U.S. ProvisionalApplication No. 61/975,699, filed Apr. 4, 2014, as well as U.S.application Ser. No. 14/065,265, which published as U.S. 2014/0120537,which claims the benefit of U.S. Provisional Application No. 61/719,491,filed Oct. 29, 2012, as well as U.S. application Ser. No. 14/836,390,which published as U.S. 2016/0059234, and which claims the benefit ofU.S. Provisional Application No. 62/042,079, filed Aug. 26, 2014, allexpressly incorporated herein by reference.

For example, as described in the above-cited references, target CTCs canbe flowed through a microfluidic channel comprising a surface, such asglass (FIG. 1 , step 125). The surface can comprise a binding moiety, towhich the CTCs of interest attach (EpCAM schematic binding illustratedin FIG. 1 , step 125). The surface can comprise a non-foulingcomposition, such as a lipid composition, a bioactive composition and/orfunctional groups, which lessens the binding of non-specific particles.The purity of the CTCs of interest is therefore enriched by reducing thebinding of non-specific particles. Once the CTCs of interest arecaptured on this surface, which can comprise a lipid bi-layer, forexample, they can be washed and stained with a panel of antibodies usinga gentle sweeping force to maintain cell integrity (FIG. 1 , step 127through step 130). The force may be, for example, a shear of airbubbles, a shear of air foams, a shear of emulsive fluid, ultrasonicvibrations or an oil phase. In one specific example, a foam compositioncomprising air bubbles is flowed over the surface to remove bound cellsand/or non-fouling compositions (FIG. 1 , step 127). In another example,as described in PCT/US2012/044701 and U.S. application Ser. No.14/128,354, which published as U.S. 2014/0255976, and which claims thebenefit of U.S. Provisional Application No. 61/502,844, filed Jun. 29,2011 and U.S. Provisional Application No. 61/606,220, filed Mar. 2,2012, all expressly incorporated herein by reference, a “releasable”composition acts to lubricate the surface so that only low flow shearstress is required to remove or release non-specific cells or bloodcomponents from the surface coating.

By way of example, target CTCs are released for imaging and analysis byflowing a foam across the microfluidic surface, which enhancesefficiency and viability of the cells, as described inPCT/US2015/023956, which claims the benefit of U.S. ProvisionalApplication No. 61/973,348, filed Apr. 1, 2014 and U.S. ProvisionalApplication No. 61/975,699, filed Apr. 4, 2014, all expresslyincorporated herein by reference.

In more general, less sophisticated examples, peripheral blood can beenriched for nucleated cells by using RBC lysis buffer in conjunctionwith positive immunomagnetic selection. Erythrocytes can be lysed byadding RBC lysis buffer, mixing by inversion, and incubating.

Another example of an enrichment process uses a highly overexpressedcell surface biomarker with high specificity and sensitivity for CTCs,such as the epithelial cell adhesion molecule (EpCAM). The CELLSEARCHSYSTEM® (Veridex) utilizes anti-EpCAM antibody-coated magneticnanoparticles to capture and enrich CTCs, followed by cytokeratinimmunostaining. The ADNATEST® (AdnaGen AG, Germany), anothercommercially available system for CTC detection, adopts a similarimmunomagnetic approach by using anti-EpCAM and Mucin 1 (MUC1)conjugated magnetic beads. More recently, “CTC chips” based onanti-EpCAM antibody-coated microfluidics chip were developed for CTCdetection and enrichment (Nagrath et al., Nature 2007, 450:1235-9). Thepatent applications referenced above address non-specific binding ofblood cells with anti-EpCAM antibody.

Next a staining process (step 130) is applied to the sample. In somecases, immunological methods, whereby antibodies directed tocharacteristic cellular constituents, can be used to stain cells ofinterest. Cell staining can be performed using monoclonal antibodies,which recognize specific cell types and features within a population ofcells. The antibodies may be directly labeled with a fluorescentcompound or indirectly labeled using, for example, a fluorescent labeledsecond antibody which recognizes the first antibody. A panel ofantibodies may be used to analyze a cell population in a multi-markerimaging approach. For example, different antibodies may be labeled withdifferent colors and subsequently imaged. In some instances, amulti-marker imaging approach may increase the sensitivity of detectionof CTCs.

Detecting and enumerating CTCs in bodily fluid samples is based on thepremise that generally cells of epithelial origin are defined as nucleicacid⁺, CD45⁻ and cytokeratin⁺ (CK). Immunocytochemical staining for anynumber of different cytokeratins (CKs) can be performed withfluorescently-conjugated antibodies or antibody fragments. Cells may befixed in ice-cold methanol, rinsed in PBS, and incubated with ananti-cytokeratin antiserum or a monoclonal antibody or antibody fragmentdirected against pan-cytokeratin (inclusive of all types ofcytokeratins), class I or II cytokeratins, or anti-individualcytokeratin isotypes (e.g., cytokeratin 1 to cytokeratin 20), or acombination of any number of cytokeratin isotypes.

Cells may also be incubated with another first (primary) antibody suchas CD45 against WBCs and/or a second antibody against the primary CD45.The sample can then be counterstained with 0.5 μg/ml DAPI in PBS at roomtemperature for 10 min, and mounted in glycerol-gelatin.

Specimens may be fixed in neutral, buffered formaldehyde and thenpermeabilized (step 135). Alternatively, slides can be dried andcover-slipped with a cellulose triacetate film or mesh, anti-fade. Instep 135, the total number of cells applied per slide can be in therange of 100 to 1.5×10⁶. An adhesive area on the slides may consist ofone to three separate circles for image analysis totaling 100 to 530mm².

The staining process includes at least two stains of different colors:cell nuclei in the biological fluid specimen are stained a first color,and cytoskeletal cell features in the biological fluid are stained asecond color. Optionally, white blood cells or other non-target cells inthe biological fluid can be stained a third color. One or both of thesestains is configured to bind preferentially to the cells of interest,e.g., using an antibody that specifically recognizes and binds tocell-surface markers or cytokeratin, for example. In someimplementations, cell nuclei can be stained with a first bio-conjugateddye that is configured to provide a first color when imaged and to bindnucleic acids in the cell nuclei of the target cells. Cytoskeletal cellfeatures can be stained with a second bio-conjugated dye configured toprovide a second color when imaged and configured to bind tocytoskeletal cell features of the target cells. In particular, thesecond stain can include an antibody or antibody fragment that binds tocytoskeletal cell features, such as cytokeratin though directimmunofluorescence. The antibody may be conjugated to a fluorescentprotein, a second antibody, or other fluorescent chemical compound thatcan re-emit light upon excitation with light. In this way, twoantibodies may be used to achieve an amplifying effect through indirectimmunofluorescence. In recognizing cytoskeletal cell features such ascytokeratin, the second stain may mark any number of cells having acytoskeleton. This includes and is not limited to epithelial cells,endothelial cells, endothelial progenitor cells, ‘cancer stem cells’ anddisseminated tumor cells, for example. White blood cells can be stainedwith a third bio-conjugated dye, such as Green Fluorescent Protein(GFP), configured to provide a third color when imaged and configured tobind to human leukocyte antigens. Indirect immunofluorescence can alsobe used with the third stain or bio-conjugated dye to amplify thesignal.

Referring now to FIG. 2 , in one particular implementation, the firststain can include a nuclear stain such as DAPI(4′,6-diamidino-2-phenylindol). A CTC should stain positive for anucleic acid dye, such as DAPI, showing that the nucleus is containedwithin the cytoplasm and smaller than the cytoplasm. The presence of anucleus indicates that the cell is not a red blood cell, which isa-nuclear.

The second stain can include one or more dye-conjugatedanti-cytokeratins (CKs). These may comprise monoclonal antibodiesspecific for cytokeratin that are conjugated to allophycocyanin (APC),phycoerythrin (PE) or any number of commercially available fluorescentmolecules such as ALEXA FLUOR® or DYLIGHT® dyes. In a particular exampleshown in FIG. 2 , the second stain includes an antibody that is specificfor cytokeratin (CK) (small oval) and conjugated to ALEXA568®, a smallmolecular organic dye with fluorescent red emission spectra, thuscapable of marking and differentiating epithelial cells. A CTC shouldstain positive for ALEXA568®-conjugated anti-cytokeratin, be round, ovalor polygonal with an intact membrane and at least about 4 μm in size.

The third stain can include anti-CD45, a monoclonal antibody specificfor CD45, an antigen present on the surface of leukocytes, conjugated toGreen Fluorescent Protein (GFP), for example, or any number ofcommercially available organic dyes such as ALEXA488® or DYLIGHT488®,with fluorescent green emission spectra, by way of illustration. A CTCshould not stain positive for CD45, as this stain recognizes an antigenpresent on leukocytes, and CTCs cannot be white blood cells. Whileparticular dyes are discussed, other similar dyes known in the art arealso contemplated.

The first color can be red or orange, the second color can be blue, andthe third color can be green, although other color combinations arepossible. The dyes can be fluorescent dyes that luminesce under light,e.g., UV light or visible light or infrared light, applied during theimaging process. Alternatively, the dyes can be absorptive dyes.

Once the stains are applied, the sample is transferred to an observationslide 20 (FIG. 1 , step 135). For example, the slide 20 can include afilter 22 (see FIG. 1 and FIG. 3 ), such as a porous membrane or mesh,and the sample can be dispensed onto the filter so that the filtercaptures candidate cells, e.g., candidate CTCs, while permitting otherfluid to flow through the filter. The filter might also capture othercells, e.g., white blood cells or other non-target cells. The filter 22can be mounted on top of the observation slide 20. The observation slide20 can be glass, plexiglass, or similar suitable material known in theart. The filter 22 can be about 5˜25 mm in diameter with an average poresize (e.g., spacing of the mesh) up to 10 μm, e.g., from 1 to 3 μm orfrom 2 to 5 μm. The average pore size can be less than 2 μm. The filter22 can be a plastic, e.g., polycarbonate.

The sample can now be analyzed (FIG. 1 , steps 140-150). In particular,candidate cells, e.g., candidate CTCs, can be identified, withoutrequiring input from a technician, using the system 200 (see FIG. 3 )discussed below. The slide 20 can be placed for imaging (step 140), thesample can be imaged (step 145), and the images analyzed to identifycandidate cells (step 150).

FIG. 3 is a schematic diagram of a system for identifying CTCs.Referring to FIG. 3 , the system 200 includes an imaging microscope 210and at least one computer 250 that can be configured to control thecapture mechanism of the microscope, control the relative motion betweenthe stage and microscope, e.g., in X, Y and/or Z directions, and/orcontrol activation of the lights source and/or movement of the opticfilters to excite and capture fluorescent lights at differentwavelengths. The computer 250 is also configured to analyze images fromthe microscope 210 and identify candidate cells, e.g., candidate CTCs,in the sample.

The imaging microscope 210 includes a digital camera 212 and opticalcomponents 214, e.g., lenses and the like, to focus the camera 212 on aspot on a slide held on a stage 220. The stage can be undergo motorizedmovement in the X, Y and/or Z directions as controlled by the computer250. The digital images captured by the microscope 210 have at least twocolor channels, e.g., three channels, e.g., a red channel, a greenchannel and a blue channel. Each color channel can correspond to thecolor generated by one of the dyes, although an exact wavelengthcorrespondence is not required. The resolution and magnification of theimaging microscope can be selected such that an individual pixelcorresponds to 0.3 to 1.3 μm on a side, e.g., about 0.648 μm². In oneexample, the imaging microscope 210 can use a 10× objective lens and adigital camera configured to generate a digital image of 1392×1040pixels with three color channels and 12 bits per channel per pixel.

The camera can be coupled to or include a memory 232 to store digitalimages from the camera 212. The memory 232 can be part of a controller230, e.g., a general purpose computer running an application, forcontrolling the microscope 210.

The stage 220 can be supported by an actuator 222, e.g., a three-axisactuator configured to move the stage 220 along two perpendicularhorizontal axes that are parallel to the plane in which the slide 20 isheld and a vertical axis that is perpendicular to the plane.Alternatively, the actuator 222 could move the camera 212 and opticalcomponents 214 while the stage 220 remains fixed in place.

The actuator 214 can be coupled to the controller 230. The controller230 can be configured to create relative motion between the stage 220and camera 212 so as to automatically scan the area imaged by themicroscope 210 across the slide 20 and to control the timing of captureof images by the camera 212 so as to generate an array of digital imagesthat cover the area in which the sample is disposed, e.g., the filterarea 22. The controller 230 can be configured to permit an operator toconfigure the microscope 210 or adjusting scanning parameters.

The imaging microscope 210 can also include a light source 240. Assumingthat the stains include fluorescent dyes, then the light source 240generates light at a wavelength appropriate to cause the dyes tofluoresce, e.g., UV to visible to infrared light. Thus, the imagingmicroscope 210 can produce digital images of the stains, e.g., thenuclear, cytokeratin, and CD45 stains. Assuming fluorescent dyes areused and that the filter 22 is composed of a material that does notfluoresce under the light from source 240, the filter should not show upin the digital image. Alternatively, if passive dyes are used, the lightsource 240 can generate white light.

The digital images are then transferred to the computer 250 for storageand analysis. The digital images can be stored and/or transmitted in alossless format such as tiff. For example, the computer 250 can becoupled to the microscope 210, e.g., by a serial bus connection such asa USB connection, or a network such as an Ethernet or the Internet, andthe computer 250 can be configured to automatically retrieve the digitalimages from the memory 232. Alternatively, the computer 250 itself couldprovide the controller 230; in this case the memory 232 could be part ofthe computer 250. Alternatively, the memory 232 could be a portabledevice, e.g., a Flash drive, which can be physically removed by theoperator from the microscope 210 and inserted in the computer 250, whichmay be a distinct computer separate from the computer that operates themicroscope.

In any case, the computer 250 is configured to receive the digitalimages and analyze them to identify candidate cells, e.g., candidateCTCs and/or white blood cells (WBCs). In general, digital images of thecandidate cells identified by the computer 250 would need to be reviewedby a technician to confirm that each candidate is, in fact, a cell ofinterest, e.g., a CTC. However, by automatically rejecting a largepercentage of any extraneous objects from the digital image, the numberof candidates for a technician to evaluate can be significantly reduced,thus increasing efficiency and reducing time needed to generate a testresult.

Referring to FIG. 4 , the computer-implemented process to identifycandidate cells includes three main steps. First, the computer 250identifies overlapping “blobs” (i.e., connected regions of pixels havinga minimum intensity) in two color channels of the digital image (seesteps 302 a and 302 b). The first color channel can correspond to thefirst color, e.g., to the staining of the nucleic acids in the cellnuclei of the target cells, and the second color channel can correspondto the color of the second dye, e.g., to the staining of thecytoskeletal cell features of the target cells. Second, the computer 250subjects a shape generated from a combination of overlapping blobs toone or more shape tests (step 304). This can exclude image artefacts aswell as some types of cells. Third, the computer 250 can exclude shapesbased on an evaluation of a third color channel (step 306). The thirdcolor channel can correspond to the third color, e.g., to the stainingof white blood cells. This can exclude white-blood cells. If priorprocesses, e.g., enrichment, are sufficiently effective at removingwhite-blood cells from the sample, then this third step could beoptional. Each of these steps will be discussed in greater detail below.

Returning to FIG. 3 , as the computer 250 identifies the candidatecells, it stores identifying information for each candidate cell. Forexample, the computer 250 could insert a tag, e.g., a bookmark, into thedigital image, could store the coordinate of the candidate cell withinthe digital image in a database, or could clip a portion of the digitalimage corresponding to the candidate cell and save that portion in aseparate file or database.

The identifying information, and the digital image if needed, can beforwarded to a classifier for a post-process or off-line inspection todetermine whether each candidate should be classified as a target cell.The identification of candidate cells can also be performed by theclassifier.

The classifier can be a human technician who will inspect the digitalimage. For example, the computer 250 can be coupled to a computer 270 bya network 260, e.g., a local area network (LAN) or the Internet.

A technician can use the computer 270 to review the portion of thedigital image corresponding to each candidate cell, e.g., views theportion of the digital image, and determines whether the candidate is,in fact, a target cell, e.g., a CTC. For example, the computer 270 maybe configured to automatically receive the identifying information,determine a portion of the digital image based on the identifyinginformation, and display in a controlled sequence the determined portionon a display of the computer 270 to the technician. For example, thecomputer 270 can receive the coordinates of the candidate cell, select aportion of the digital image centered at the coordinates, and thenautomatically display the selected portion. This can reduce the need forthe technician to search through the digital image for the candidates.In another implementation, the technician may receive a list ofcandidates, with each element in the list linked to a portion of thedigital image. Selection of an element from the list by the user cancause the computer to present the corresponding portion of the digitalimage to the technician.

Alternatively, if in the future it becomes possible to entirely automatedetermination of whether candidate cells are target cells, this taskcould be performed by providing the proper instructions to a computer272. In this case, the classifier is the computer 272. For example, thecomputer 250 can be coupled to a computer 272 by the network 260.

In some implementations, the computer 270 or 272 is configured to countthe number of cells that are determined to be target cells, e.g., by thetechnician or automatically. This count can be used to generate a score,e.g., a total number of CTCs, a percentage, a ratio relative to ahealthy individual, or the like. In other implementations, the score maybe generated from a regression equation which include, in addition toCTC counts, other risk factors such as age, gender, body mass index,family cancer history, alcohol usage, physical activities or other lifestyles, to name a few. Eventually test results are returned to theentity, e.g., the doctor, who ordered the test. The test results couldinclude the score, the portions of the digital image corresponding tothe target cells, or both. Based on these test results, a doctor could,for example, determine the likelihood of cancer occurrence or recurrenceby comparing the score with a statistically-determined score ofcirculating epithelial cells from a group of tumor-free patientcontrols. Based on this comparison, a doctor could assign a likelihoodof cancer occurrence or recurrence when the total score obtained exceedsa pre-determined value based on statistical averages of circulatingepithelial cell counts from healthy subjects.

The score can also serve to screen for undetected cancer in healthyindividuals, diagnose cancer in patients with symptoms or detect aputative change in patient status. In one implementation, the scoreassists in determining an appropriate course of treatment for aggressiveor indolent forms of cancer. In this regard, DNA within the CTCs sampledfrom the patient can be sequenced using next-generation sequencingtechnologies to identify cancer driver mutations. The mutations can beevaluated against panels of genetic markers that are understood tocorrelate with particular targeted therapies. Targeted therapies attackspecific types of cancer cells with less harm to normal cells. Oneexample is HERCEPTIN® (Genentech, USA), an antibody pharmaceutical,which targets the human epidermal growth factor receptor 2, HER2, whichis over-expressed on the surface of cancerous ovarian and breast tissue.

The CTCs within a patient's sample can be subjected to other genetictests. In the case of lung cancer, a blood draw to assess CTCs benefitspatients that are too ill to provide a lung tumor biopsy. Based on asimple blood draw and CTC sequencing, genetic tests of the CTC DNA canreveal whether a patient has non-small cell lung cancer, which harborsepidermal growth factor receptor (EGFR) gene mutations. Knowing thegenetic makeup of a tumor helps physicians decide whether a patientwould benefit from regular chemotherapy or a targeted anti-cancer agentlike TARCEVA®, which inhibits particular activated mutated forms of theEGF receptor.

By detecting, and enumerating CTCs, and further sequencing the DNA inCTCs, doctors can better understand a patient's particular cancersubtype. This can inform therapy choices and improve outcomes. In thecase of breast cancer, for example, 70% of tumorous cells express anoverabundance of hormone receptors, which bind estrogen or progesteroneand stimulate cell growth. These tumors are best treated with hormonaltherapy. Among other breast cancer subtypes, about 20%, have anoverabundance of receptors that bind human epidermal growth factor 2(HER2). These cells are best attacked with drugs like TYKERB® (Novartis,USA) and HERCEPTIN® (Genentech, USA), which target the HER2 receptor.Other breast cancer cells, roughly 10%, are “triple negative” meaningthey do not have an overabundance of any of these receptors, and mayinstead harbor mutations in the BRCA tumor-suppressor genes. HERCEPTIN®(Genentech, USA) would not be appropriate treatment for these tumors,and treatment of patients without the benefit of information provided bythe detection, quantification and sequencing of CTCs could result insuboptimal outcomes and potentially increased mortality.

Turning to the process implemented by computer 250, FIG. 5 is a flowchart of a computer-implemented process of identifying overlapping“blobs” in the digital image. To identify each “blob,” the computeridentifies connected regions of pixels that have a minimum intensity.The connected regions are contiguous areas of adjacent pixels in thedigital image. Each connected region can be subject to a size test,e.g., whether the region has a size, e.g., in total number of pixels,between an upper threshold and a lower threshold. The upper thresholdand lower threshold can be predetermined based on the resolution of theimage (i.e., number of pixels per micron) to correspond to physicalsizes (e.g., in microns) that would correspond to CTCs.

In particular, to identify each “blob” in the first color channel, thecomputer identifies first connected regions of pixels of a minimum firstintensity in the first color channel (step 312).

Whether the first connected region has a minimum first size isdetermined (step 314). For example, the total number of pixels in thefirst connected region can be calculated. If the first connected regionhas a total number of pixels less than a first lower threshold, e.g.,fifty pixels, it will be eliminated as a candidate. Since a cell nucleushas a minimum size, if the first connected region is too small, thisindicates that the “blob” in the first channel is not a cell nucleus andthus not a candidate.

In addition, whether the first connected region has a maximum first sizecan be determined. For example, if a first connected region has a totalnumber of pixels greater than a first upper threshold, e.g.,fifteen-hundred, e.g., one thousand pixels, it will be eliminated as acandidate. Since a cell nucleus has a maximum size, if the firstconnected region is too large, this indicates that the “blob” is not acell nucleus and thus not a candidate.

A similar process can be performed for the second color channel. Toidentify each “blob” in the second color channel, the computeridentifies second connected regions of pixels of a minimum secondintensity in the second color channel (step 316).

Whether the second connected region has a minimum second size can bedetermined (step 318). For example, the total number of pixels in thesecond connected region can be calculated, e.g., by rasterizing throughthe image after the region has been determined and counting the pixelsthat are marked as being in the second connected region. If the secondconnected region has a total number of pixels less a second lowerthreshold, e.g., one-hundred pixels, it will be eliminated as acandidate. Since a cell has a minimum size, if the first connectedregion is too small, this indicates that the “blob” in the second colorchannel is not a cell and thus not a candidate.

In addition, whether the second connected region has a maximum secondsize can be determined. For example, if a second connected region has atotal number of pixels greater than a second upper threshold, e.g., onethousand five hundred pixels, it will be eliminated as a candidate.Since a cell has a maximum size, if second connected region is toolarge, this indicates that the “blob” is not a cell and thus not acandidate.

Instead of comparing the number of pixels to a threshold number, thenumber of pixels could be divided by a resolution, e.g., number ofpixels per unit area, and then compared to thresholds that representsizes in units of area, e.g., in square microns.

One implementation of identifying connected regions of pixels that havea minimum intensity is to convert a grayscale image into a binary imagethrough thresholding (e.g., those pixels having lower than the minimumintensity are set to zero, those pixels having greater than the maximumintensity are set to one). The process examines pixels of the binaryimage, e.g., running from top to bottom and left to right in the digitalimage, determining whether the pixel is adjacent another pixel that hasalready been assigned to a blob. If the pixel is adjacent a pixel froman existing blob, the pixel can be assigned to the already identifiedblob. Otherwise a new blob data record will be created and stored (e.g.,the pixel is assigned a new blob record). The minimum intensity can beselected through empirical research to be a value that distinguishescells from noise, e.g., by an operator adjusting the minimum intensityin each color channel while a sample image is being displayed anddetermining by visual inspection whether the threshold reliablygenerates a reliably distinguishes cells from noise.

Additional image filtering techniques, e.g., based on intensitysmoothing, can be applied to the image. The noise filtering can avoidfalse “on” pixels and thus prevent creation of a new blob data record,e.g., if the size of the blob is sufficiently small. The noise filteringcan also avoid false “off” pixels to improve the calculation of thetotal number of pixels in the second connected region.

Another implementation of identifying connected regions of pixels thathave a minimum intensity is a maximally stable extremal regions (MSER)algorithm. In this technique, pixels in the digital image are sorted inintensity order. The sorted pixels are placed one-by-one to a blankimage to grow blobs. In particular, the process includes a try-catchclause including a for-loop with sorted intensity index as the iterationvariable. Inside this for loop, ending conditions are first checked tosee if an image background intensity has been reached or not. The endingconditions can include two aspects: if the number of blobs in the imagereaches 20, and if the largest blob in the image reaches a size of 3000pixels. If, for the given iteration with the given pixel intensity atthe given image position, the ending condition is not reached, thatiteration will go through by placing that given pixel intensity at thatgiven image position.

After the placement of a pixel, adjacent pixels are checked to see ifthere are already placed pixels present in the image or not. If thereare, existing blob data records will be updated for this pixel'splacement (e.g., the pixel is assigned to a blob that has already beennoted), otherwise, a new blob data record will be created and stored(e.g., the pixel is assigned a new blob record). Such pixel placementwill continue until the ending condition is reached. When the endingcondition is reached, the iteration will stop, and the operation willexit for loop and the try-catch clause is completed. Maximally stableextremal regions are discussed in J. Matas et al., “Robust Wide BaselineStereo from Maximally Stable Extremal Regions,” Electronic Proceedingsof The 13th British Machine Vision Conference (2002).

In some implementations, the digital image is divided into a pluralityof regions, and each region is analyzed. The number of pixels for theregions can be selected such that the region can be larger than theexpected size of the target cells, e.g., by about a factor of 3 to 10.For example, the digital image can be divided into rectangular regionsthat are, for example, 50 to 200 pixels on a side, e.g., into 100×100pixel squares. Assuming that pixels are 0.648 μm on a side, this 100×100pixel region can represent a 64.8 μm×64.8 μm area on the observationslide. Since CTCs measure between 8 μm to 20 μm, the region is largeenough for the CTC to fit completely within the area.

In some implementations, if a connected region is identified asincluding more than a threshold number of pixels, a new re-centeredregion is selected that is centered on the connected region, and theidentification of overlapping “blobs” is performed for the re-centeredregion. The re-centering can include forming an image that includesportions from multiple adjacent regions of a segmented image array.

Next, whether each first connected region overlaps a second region isdetermined (step 252 a). Since a circulating tumor cell includes both anucleus and cytoskeleton, presence of a connected region in one of thefirst or second color channels without a corresponding connected regionin the other of the first and second color channels indicates that theconnected region is not part of and therefore not a target cell andtagged as such. Thus, the computer 250 can determine regions of overlapof the “blobs” in the first color channel and the second color channel;if there is no overlap then a blob can be rejected as a candidate.

In some implementations, the second connected region can be consideredto overlap the first connected region if the second region overlapsand/or fits within the first connected region. In some implementations,the first connected region is required to surround the second connectedregion, e.g., the first connected region forms a continuous annulusaround the second connected region. Binary filtering of two of thechannels, e.g., the blue and red channels, can be performed as a part ofdetermining the overlap between blue blob and red blob.

The potential candidates indicated by overlapping connected regions fromthe first and second color channels can now be subjected to one or moreshape tests.

A first shape test is to determine whether an aspect ratio of theoverlapping connected region falls within threshold aspect ratio range.The target cells, e.g., CTCs, are generally not elongated. Assuming theaspect ratio is the ratio of the shorter measurement to the longermeasurement, then if the overlapping connected region has an aspectratio below a threshold, this indicates that the overlapping connectedregion is too elongated and is not a candidate.

In some implementations, to determine the aspect ratio, as shown in FIG.6A, a combined region that is the union of the first connected regionand the overlapping second connected region can be determined, and thenthe outer boundary of the combined region can be determined. The outerboundary can be the set of pixels on the outer perimeter of the combinedregion.

Next, as shown in FIG. 6B, a major axis for the combined region can becalculated. The major axis can be the line segment connecting the twopixels on the boundary that are farthest apart. For example, todetermine the major axis, perform a function that for each pixel on theboundary, the distance between that pixel and each other pixel on theboundary can be determined; the pair with the greatest distance providethe two pixels that define the major axis.

Then, as shown in FIG. 6C, the minor axis can be calculated for thecombined region. The major axis splits the combined region into twohalves. The minor axis can be the longest line segment that isperpendicular to the major axis that connects two pixels on the boundaryon opposite sides of the major axis. For example, to determine the minoraxis, for each pixel on the boundary on one side of the major axis, thedistance between that pixel and the pixel on the boundary on the otherside of the major axis on a line perpendicular to the major axis can bedetermined; the pair with the greatest distance provide the two pixelsthat define the minor axis.

The aspect ratio can be calculated as the ratio of the length of theminor axis to the length of the major axis. The aspect ratio can becompared to a threshold value. For example, if the aspect ratio is lessthan 0.4, e.g., less than 0.35, e.g., less than 0.3, then theoverlapping connected region can be eliminated as a candidate cell.

A second shape test is to compare the fill factor of the overlappingconnected regions. This provides an alternate way to detect an elongatedregion or to eliminate extreme irregularity in shape.

In some implementations, referring to FIG. 7 , to determine the fillfactor, a bounding box is established around the combined region. Thebounding box is a rectangle with an upper and lower boundary that matchthe uppermost and lowermost pixels of the combined region, and a rightand left boundary that match the rightmost and leftmost pixels of thecombined region.

The number of pixels within the outer boundary of the combined regioncan be counted, and the number of pixels that are outside the combinedregion but inside the boundary box (also known as the “extent”) can becounted. The ratio of the number of pixels within the outer boundary tothe number of pixels in the extent is compared to a fill ratiothreshold. If the ratio is less than the fill ratio threshold, thecombined region is not a candidate. The fill ratio threshold can beabout 0.4 to 0.85, e.g., 0.60.

Assuming that all of the prior tests are passed, the combined region canbe identified as a candidate, although it is possible for non-targetcells and white blood cells to pass these tests. Therefore, anadditional process can be used to screen white blood cells. In general,this step includes determining an intensity ratio between two differentchannels in the combined region.

One of the channels can be the second channel. Another of the channelscan be the third channel. In this case, a third connected regions in thethird channel are determined, e.g., as described above for the first andsecond channels, and regions where the second connected regions andthird connected regions overlap can be determined. The intensity valuesin the two channels in the region of overlap can be used to determinethe intensity ratio. The spatially overlapping second connected regionsand third connected regions for which the intensity ratio does not meetan intensity ratio can be eliminated as candidates.

In brief, a white blood cell should contain a greater amount of thethird stain, and thus should show greater intensity in the third colorchannel. Thus, candidate regions that also show a strong intensity inthe third color channel can be rejected.

In some implementations, this test can be performed by calculating anaverage intensity I2 of the second connected region. In addition, athird connected region of pixels of a minimum third intensity in thethird color channel can be identified, e.g., using one of the techniquesdescribed above for the first and second connected regions. The averageintensity I3 of the third connected region is also calculated.

In some implementations, the average background intensity within a colorchannel, e.g., the intensity within the extent region or within aselected image subregion that includes the combined area, is calculated.This average background intensity for the color channel is subtractedfrom the average intensity of the connected region in the channel toprovide an adjusted average intensity.

The following formula can be used to determine whether the cell is acandidate cell, e.g., a CTC (Group 1), ambiguous (Group 2), or a whiteblood cell (Group 3).

$\begin{matrix}{\frac{I\; 2}{{I\; 2} - {I\; 3}} \leq 2} & {{Group}\mspace{14mu} 1} \\{\frac{I\; 2}{{I\; 2} - {I\; 3}} > {2\mspace{14mu}{and}\mspace{14mu}\frac{I\; 2}{I\; 3}} > 1} & {{Group}\mspace{14mu} 2} \\{\frac{I\; 2}{I\; 3} < 1} & {{Group}\mspace{14mu} 3}\end{matrix}$where I2 and I3 are the averages or adjusted averages discussed above.

The combined regions corresponding to Group 3 can be rejected; thoseregions that satisfy I2/I3≥2 can be marked as candidates for evaluationby the technician.

In review, using algorithms described herein, the computer 250determines overlapping first connected regions and second connectedregions from first and second color channels in the digital image. Anaspect ratio of the spatially overlapping first connected regions andsecond connected regions is then determined based on one or more colorchannels. One or more spatially overlapping first connected regions andsecond connected regions for which the aspect ratio meets a thresholdare then determined. Portions of the image corresponding to anidentified spatially overlapping first connected region and secondconnected region are then displayed to an operator as candidates forclassification.

FIG. 8 is a flow chart illustrating considerations in determiningwhether an object in an image should be classified as a candidate. Thisflow chart is not necessarily in the same order as the operations bysoftware on computer 250. Whether the candidate region in the image hasthe morphology of a cell is determined, e.g., using a shape testdiscussed above. If the candidate region does not have the morphology,i.e., does not pass a shape test, it is removed from consideration ashaving poor morphology. If the candidate region has the morphology,i.e., passes a shape test, it can continue to be treated as a candidatefor classification. Assuming there are one or more candidate regionsthat have the morphology of a cell, whether this shape is reproduced inthe image of the same is determined. If the shape is not reproduced,i.e., only a single candidate is identified, then the candidate regionis treated as an image artifact. On the other hand if the shape isreproduced, i.e., a plurality of candidate regions are identified, thenthe portions of the image corresponding to the plurality of candidateregions are continued to be treated as candidates for classification infurther analysis.

EXAMPLES

Detection of CTCs finds important application in diagnosis and prognosisof cancer. The presence in peripheral blood of CTCs expressing on theirsurface the ‘Epithelial Cell Adhesion Molecule’ (EpCAM, a pan-epithelial(all-inclusive) differentiation antigen that is expressed on almost allcarcinomas) is associated with decreased progression free survival anddecreased overall survival in patients treated for metastatic breastcancer. CTCs that express EpCAM are abbreviated as “EpCAM+ CTCs.” Forsome platforms, an EpCAM+ CTC count of 5 or more per roughly 7.5 ml ofblood is predictive of shorter progression free survival and overallsurvival.

In another application, 2 to 10 mL of blood may be processed per sample,and the number of circulating tumor cells identified may be used topredict the risk or recurrence of diseases. In yet anotherimplementation, the number of CTCs identified from the blood may be justone of several variables in a regression equation which contain otherrisk factors, other than CTCs, as variables.

In some implementations, a CTC should stain positive for CK-ALEXA568® (ared dye), be round, oval or polygonal with an intact membrane and atleast 4 μm in size. A CTC should not stain positive for CD45-ALEXA488®(a green dye), as this stain recognizes an antigen present onleukocytes, and CTC cannot be white blood cells. A CTC should also stainpositive for the DAPI nuclear stain (a blue dye), indicating that thenucleus is contained within the cytoplasm and smaller than the cytoplasmby at least 30%. The presence of a nucleus indicates that the cell isnot a red blood cell, which is a-nuclear.

CTCs are shed by tumor masses into the bloodstream. Tracking andenumerating CTCs in peripheral blood early can alter treatment regimensand possibly slow metastasis. CTCs are rare and difficult to classifyamong other cells. As shown in FIG. 6 , a field view of schematic cellsderived from peripheral blood are stained before optical viewing. Imagegalleries of cells are presented for review, where each gallery containsa CK-ALEXA568® (cytokeratin) column, a DAPI nuclear counterstain, and aCD45-ALEXA488® stain for WBC (or other non-target cells), for example. Atechnologist can review each line and select those cells that qualify astumor cells according to specific criteria. In some implementations, atechnologist may interpret objects in image galleries as circulatingtumor cells (CTCs), leukocytes (WBC), squamous cells, tumor cells withleukocytes in the same frame, non-target cells or dual positive cells,for example. Miscellaneous considerations when viewing image objectsinclude a-nucleated cells, pixilated cells, detached nuclei andnon-cellular debris such as artifacts and computer noise.

Using a light control, a first image is first captured with red light, asecond image is captured with blue light and a third image is capturedusing green light.

The microscope may have a motorized platform capable of scanning thesample on the slide and capturing images in sections. Using automatedsteps, a computer-implemented process described herein interpretsoverlapping colors and shapes (circular/oval versus non-cellular). Incertain implementations, the motorized platform reorients the slide andresolves which sections have CTCs. It stores these images as sections ofinterest for review.

Large, atypical appearing cells should not be counted as CTCs.Pixilated, blurry and non-intact cells should not be counted as CTCs.Rare contaminants may appear due to over-amplification of theCK-ALEXA568® or DAPI signals. These images do not meet the criteria foraCTC.

In one implementation, an image capture apparatus (e.g., a microscope)identifies first connected regions of blue pixels of a minimum firstintensity. The connected blue pixels of a minimum first intensitycorrespond to putative WBC or CTC nuclei. Identification of connectedregions of blue pixels is performed in 100×100 pixel patches. Onceidentified, the system will re-center the patch such that the connectedregions of blue pixels are centered within the 100×100 pixel patch.

The image capture apparatus then identifies second connected regions ofred pixels of a minimum second intensity. First connected regions ofblue pixels and second connected regions of red pixels that spatiallyoverlap are identified and an aspect ratio of the spatially overlappingfirst and second connected regions is determined. One or more spatiallyoverlapping regions for which the aspect ratio meets a threshold, theregion is then identified. Portions of the image corresponding tospatially overlapping first connected regions and second connectedregions are then displayed to an operator as candidates forclassification.

Definitions

The term “circulating tumor cells” (CTCs) is used herein to indicatenucleated cells in a circulating fluid (preferably peripheral blood)that are not leukocytes.

CTCs are rare cells that have left a primary tumor to enter the bloodstream or lymphatic system. In the case of bladder cancer, CTCs candislodge from a tumor mass and enter urine. In the case of salivarygland cancer, CTCs can detach from a tumor mass and enter saliva. Thus,the methods and processes described herein define a CTC as an objectthat has a nucleus (e.g., stains positive for DAPI), has epithelial cellfeatures (e.g., stains positive for cytokeratin), and is not a leukocyte(e.g., does not stain positive for CD45). The object must be larger than4×4 μm² and have cell-like morphology.

As used herein, the term “DAPI” refers to 4′,6-diamidino-2-phenylindole,a stain with fluorescent blue emission spectra that binds strongly toA-T rich regions in DNA. When used in fluorescence microscopy, DAPIpasses through intact cell membranes in both live and fixed cells, thuscapable of marking and differentiating between nucleated and a-nucleatedcells such as red blood cells.

As used herein, the term “CK” refers to cytokeratin, which is akeratin-containing intermediate filament found in the intracytoplasmiccytoskeleton of epithelial cells. The term “anti-CK/ALEXA568®” refers tomonoclonal antibodies specific for cytokeratin that are conjugated toALEXA568®, an organic dye with fluorescent red emission spectra, thuscapable of marking and differentiating epithelial cells. Note theanti-CK/ALEXA568® can also refer to a combination of a first antibodyrecognizing CK and a second antibody conjugated to a dye such asALEXA568® that targets the first antibody. Anti-cytokeratins (CKs) canbe anti-pan (all-inclusive) cytokeratin or anti-individual cytokeratinisotypes (e.g., cytokeratin 1 to cytokeratin 20) or a combination of anynumber of cytokeratin isotypes. More generally, “anti-CK/PE” and“anti-CK/APC” refer to monoclonal antibodies specific for cytokeratinthat are conjugated to classes of organic dyes with fluorescent redemission spectra, such as, allophycocyanin (APC) and phycoerythrin (PE),for example.

As used herein, the term “CD45” refers to cluster differentiation 45, anantigen present on the surface of leukocytes. Anti-CD45-ALEXA488® is amonoclonal antibody specific for CD45 conjugated to ALEXA488®, anorganic dye with fluorescent green emission spectra, thus capable ofmarking and differentiating leukocytes, also referred to as white bloodcells (WBC). Note the anti-CD45/ALEXA488® can also refer to acombination of a first (CD45) and a second antibody (ALEXA488®) thattargets the first antibody.

As used herein, the term “bodily fluid” includes ascites, saliva, urine,synovial fluid, peritoneal fluid, amniotic fluid, cerebrospinal fluid,serosal fluid and/or spinal fluid.

The term “nuclear stain” refers to a dye compound used to indicate thepresence of a nucleus in a cell. Nuclear stains include suchintercalating dyes such as acridine orange, ethidium bromide, ethidiummonoazide, Hoechst dyes, propidium iodide and DAPI.

The term “fluorescent label”, as used herein, refers to a fluorophorethat can be covalently attached to another molecule, such as a proteinor nucleic acid, which attachment is generally accomplished by using areactive derivative of the fluorophore that selectively binds to afunctional group contained in the target molecule. Fluorescent labelsinclude, but are not limited to allophycocyanin (APC), fluoresceins(FITC), rhodamines (FAM, R6G, TET, TAMRA, JOE, HEX, CAL Red, VIC, andROX), Texas red, BODIPY, coumarins, cyanine dyes (thiazole orange [TO],oxazole yellow [YO], TOTO, YOYO; Cy3, Cy5), ALEXA FLUOR® dyes, DYLIGHT®dyes Green Fluorescent Protein (GFP), and phycoerythrin (PE).

The term “biological sample” as used herein, is used in its broadestsense as containing nucleic acids or the protein translation productsthereof. A sample may comprise a bodily fluid such as blood; the solublefraction of a cell preparation, or an aliquot of media in which cellswere grown; a chromosome, an organelle, or membrane isolated orextracted from a cell; genomic DNA, RNA, or cDNA in solution or bound toa substrate; a cell; a tissue; a tissue print; a fingerprint; cells;skin, and the like. In preferred implementations, the term refers tobiological material obtained from a subject that contains cells andencompasses any material in which CTCs can be detected. A sample can be,for example, whole blood, plasma, saliva or other bodily fluid or tissuethat contains cells. A preferred sample is whole blood, more preferablyperipheral blood, still more preferably a peripheral blood cellfraction, still more preferably CTCs isolated or enriched from blood.

The term “antibody” as used herein refers to any of a large variety ofproteins normally present in the body or produced in response to anantigen which it neutralizes, thus producing an immune response. Anantibody preferably comprises immunoglobulins of the IgG subtype.

The term “reacts specifically with” as used herein, refers to thebinding between an antibody and an antigen with a specificity (andgenerally also affinity) which is better than the binding between thesame antigen and a non-specific antibody.

A volume of blood (or other bodily secretion) necessary for analysisusing the systems and methods described herein may be equal to about orless than 25 μL, 50 μL, 75 μL, 100 μL, 0.2 mL, 0.5 mL, 1 mL, 1.5 mL, 2mL, 2.5 mL, 3 mL, 3.5 mL, 4 mL, 4.5 mL, 5 mL, 5.5 mL, 6 mL, 6.5 mL, 7mL, 7.5 mL or 8 mL, 9 mL, 10 mL, 11 mL, 12 mL, 13 mL, 14 mL, 15 mL or 16mL. The volume of blood necessary for analysis using the systems andmethods described herein may be equal to or up to 25 μL, 50 μL, 75 μL,100 μL, 0.2 mL, 0.5 mL, 1 mL, 1.5 mL, 2 mL, 2.5 mL, 3 mL, 3.5 mL, 4 mL,4.5 mL, 5 mL, 5.5 mL, 6 mL, 6.5 mL, 7 mL, 7.5 mL or 8 mL, 9 mL, 10 mL,11 mL, 12 mL, 13 mL, 14 mL, 15 mL or 16 mL. As used herein, the term“about” may refer to an amount within +/−1, 2, 3, 4, 5, 6, 7, 8, 9, or10% of a subsequently mentioned value.

For example, a sample of 11 mL of blood may also be referred to as asample of blood equal to about 10 mL. Where a range of values isprovided, it is understood that each intervening value, to the tenth ofthe unit of the lower limit unless the context clearly dictatesotherwise, between the upper and lower limits of that range is alsospecifically disclosed. Each smaller range between any stated value orintervening value in a stated range and any other stated or interveningvalue in that stated range is encompassed within the invention. Theupper and lower limits of these smaller ranges may independently beincluded or excluded in the range, and each range where either, neitheror both limits are included in the smaller ranges is also encompassedwithin the invention, subject to any specifically excluded limit in thestated range. Where the stated range includes one or both of the limits,ranges excluding either or both of those included limits are alsoincluded in the invention. The following examples are presented in orderto more fully illustrate the preferred implementations of the invention.They should in no way be construed, however, as limiting the broad scopeof the invention.

Computer Systems

One or more of the computers described above, e.g., the computer 250,include a processor 252, a memory 254, a storage device 256, and one ormore input/output interface devices 258. Each of the components 252,254, 256, and 258 can be interconnected, for example, using a system bus259.

The processor 252 is capable of processing instructions for executionwithin the system 250. The term “execution” as used here refers to atechnique in which program code causes a processor to carry out one ormore processor instructions. In some implementations, the processor 252is a single-threaded processor. In some implementations, the processor252 is a multi-threaded processor. In some implementations, theprocessor 252 is a quantum computer. The processor 252 is capable ofprocessing instructions stored in the memory 254 or on the storagedevice 256.

The memory 254 stores information within the system 250. In someimplementations, the memory 254 is a computer-readable medium. In someimplementations, the memory 254 is a volatile memory unit. In someimplementations, the memory 254 is a non-volatile memory unit.

The storage device 256 is capable of providing mass storage for thesystem 250. In some implementations, the storage device 256 is anon-transitory computer-readable medium. In various differentimplementations, the storage device 254 can include, for example, a harddisk device, an optical disk device, a solid-state drive, a flash drive,magnetic tape, or some other large capacity storage device. In someimplementations, the storage device 254 may be a cloud storage device,e.g., a logical storage device including one or more physical storagedevices distributed on a network and accessed using a network, such asthe network 260 shown in FIG. 3 . In some examples, the storage device256 may store long-term data, such as the digital images.

The input/output interface devices 258 provide input/output operationsfor the system 250. In some implementations, the input/output interfacedevices 256 can include one or more of a network of interface devices,e.g., an Ethernet interface, a serial communication device, e.g., anRS-232 interface, and/or a wireless interface device, e.g., an 802.11interface, a 3G wireless modem, a 4G wireless modem, etc. A networkinterface device allows the system 256 to communicate, for example,transmit and receive data such as the digital images, e.g., using thenetwork 260. In some implementations, the input/output device caninclude driver devices configured to receive input data and send outputdata to other input/output devices, e.g., keyboard, printer and displaydevices. In some implementations, mobile computing devices, mobilecommunication devices, and other devices can be used.

The software to carry out the image analysis and other operations of thesystem 200 can be realized by instructions that upon execution cause oneor more processing devices to carry out the processes and functionsdescribed above. Such instructions can include, for example, interpretedinstructions such as script instructions, or executable code, or otherinstructions stored in a computer readable medium.

Although an example processing system has been described,implementations of the subject matter and the functional operationsdescribed above can be implemented in other types of digital electroniccircuitry, or in computer software, firmware, or hardware, including thestructures disclosed in this specification and their structuralequivalents, or in combinations of one or more of them. Implementationsof the subject matter described in this specification, such as storing,maintaining, and displaying artifacts can be implemented as one or morecomputer program products, i.e., one or more modules of computer programinstructions encoded on a tangible program carrier, for example acomputer-readable medium, for execution by, or to control the operationof, a processing system. The computer readable medium can be a machinereadable storage device, a machine readable storage substrate, a memorydevice, or a combination of one or more of them.

The term “system” may encompass all apparatus, devices, and machines forprocessing data, including by way of example a programmable processor, acomputer, or multiple processors or computers. A processing system caninclude, in addition to hardware, code that creates an executionenvironment for the computer program in question, e.g., code thatconstitutes processor firmware, a protocol stack, a database managementsystem, an operating system, or a combination of one or more of them.

A computer program (also known as a program, software, softwareapplication, script, executable logic, or code) can be written in anyform of programming language, including compiled or interpretedlanguages, or declarative or procedural languages, and it can bedeployed in any form, including as a standalone program or as a module,component, subroutine, or other unit suitable for use in a computingenvironment. A computer program does not necessarily correspond to afile in a file system. A program can be stored in a portion of a filethat holds other programs or data (e.g., one or more scripts stored in amarkup language document), in a single file dedicated to the program inquestion, or in multiple coordinated files (e.g., files that store oneor more modules, sub programs, or portions of code). A computer programcan be deployed to be executed on one computer or on multiple computersthat are located at one site or distributed across multiple sites andinterconnected by a communication network.

Computer readable media suitable for storing computer programinstructions and data include all forms of non-volatile or volatilememory, media and memory devices, including by way of examplesemiconductor memory devices, e.g., EPROM, EEPROM, and flash memorydevices; magnetic disks, e.g., internal hard disks or removable disks ormagnetic tapes; magneto optical disks; and CD-ROM, DVD-ROM, and Blu-Raydisks. The processor and the memory can be supplemented by, orincorporated in, special purpose logic circuitry. Sometimes a server isa general purpose computer, and sometimes it is a custom-tailoredspecial purpose electronic device, and sometimes it is a combination ofthese things. Implementations can include a back end component, e.g., adata server, or a middleware component, e.g., an application server, ora front end component, e.g., a client computer having a graphical userinterface or a Web browser through which a user can interact with animplementation of the subject matter described is this specification, orany combination of one or more such back end, middleware, or front endcomponents. The components of the system can be interconnected by anyform or medium of digital data communication, e.g., a communicationnetwork. Examples of communication networks include a local area network(“LAN”) and a wide area network (“WAN”), e.g., the Internet.

Although the discussion above has focused on detection of CTCs that areepithelial cells, in principle the techniques described herein could beapplied to other kinds of cells, such as, for example, other circulatingrare cells (CRCs), disseminated cancer cells, stem cells (e.g., tumorstem cells and bone marrow stem cells), fetal cells, bacteria,endothelial cells or the like.

A number of implementations have been described. Nevertheless, it willbe understood that various modifications may be made without departingfrom the spirit and scope of the disclosure. Accordingly, otherimplementations are within the scope of the following claims.

What is claimed is:
 1. A computer program product for identifyingcandidate target cells within a biological fluid specimen, the computerprogram product tangibly embodied in a non-transitory computer readablemedium, comprising instructions to cause a processor to: receive adigital image of the biological fluid specimen; identify at least onecandidate region of pixels in the digital image having the shape of acell, wherein the instructions to identify the at least one candidateregion having the shape of a cell include instructions to identifyconnected regions of pixels of a minimum intensity having a size betweena minimum size and a maximum size and an aspect ratio that meets athreshold; and determine whether a single candidate region having theshape is identified or a plurality of candidate regions having the shapeare identified in the image; and if a single candidate is identifiedthen treat the candidate region as an image artifact, whereas if aplurality of candidate regions are identified then continue to treat theportions of the image corresponding to the plurality of candidateregions as candidates for classification.
 2. The computer programproduct of claim 1, wherein the instructions to identify at least onecandidate region having the shape of a cell comprise instructions to:identify first connected regions of pixels of a minimum first intensityin a first channel of a plurality of color channels of the digitalimage; identify second connected regions of pixels of a minimum secondintensity in a second channel of the plurality of color channels;determine first connected regions and second connected regions thatspatially overlap; determine an aspect ratio of the spatiallyoverlapping first connected regions and second connected regions basedon a color channel of the plurality of color channels; and identifyspatially overlapping first connected regions and second connectedregions for which the aspect ratio meets an aspect ratio threshold. 3.The computer program product of claim 2, comprising instructions todetermine whether first connected regions and second connected regionsthat spatially overlap include pixels having a minimum third intensityin a third channel of the plurality of color channels, and if acandidate region does not have pixels having the minimum third intensityin the third channel then continue to treat the portions of the imagecorresponding to the candidate region as a candidates forclassification.
 4. The computer program product of claim 3, comprisinginstructions to treat a candidate region having pixels having theminimum third intensity in the third channel as a leukocyte or whiteblood cell.
 5. The computer program product of claim 2, wherein theplurality of channels comprises a first color and a second color,wherein the first color is blue, and the second color is red or orange.6. The computer program product of claim 1, comprising instructions toprovide portions of the image corresponding to at least one of theplurality of candidate regions to a classifier as a candidate forclassification.
 7. The computer program product of claim 1, comprisinginstructions to for each candidate region of at least one of the one ormore candidate region, determine whether the portion of the imagecorresponding to the candidate region includes more than a thresholdnumber of intensity levels, and if the portion of the imagecorresponding to the candidate region includes more than the thresholdnumber of intensity levels then continue to treat the portion of theimage as a candidate for classification.
 8. The computer program productof claim 7, comprising instructions to cause a processor to: if theportion of the image corresponding to the candidate region includes atmost the threshold number of intensity levels, then treat the candidateregion as a pixilation.
 9. The computer program product of claim 7,wherein the threshold number is three.
 10. The computer program productof claim 8, wherein the instructions to identify one or more candidateregions comprise instructions to: identify first connected regions ofpixels of a minimum first intensity in a first channel of a plurality ofcolor channels of the digital image; identify second connected regionsof pixels of a minimum second intensity in a second channel of theplurality of color channels; determine first connected regions andsecond connected regions that spatially overlap; determine an aspectratio of the spatially overlapping first connected regions and secondconnected regions based on a color channel of the plurality of colorchannels; and identify spatially overlapping first connected regions andsecond connected regions for which the aspect ratio meets an aspectratio threshold.
 11. The computer program product of claim 10, whereinthe plurality of channels comprises a first color and a second color,wherein the first color is blue, and the second color is red or orange.12. The computer program product of claim 8, comprising instructions toprovide the portion of the image corresponding to the candidate regionto a classifier as a candidate for classification.